Speech recognition systems typically aim to convert a voice input from a user into a natural language transcription. Speech recognition systems may use a language model in the process of determining the words that a user has spoken. For example, a language model may include statistical information about language that can be used to estimate which words are likely to occur in a language sequence.